Live Text Access.
Training for real time intralingual subtitlers.
The introduction to this Unit has already explained what respeaking is.
The introduction to this element has already explained
the Psycho-cognitive skills a respeaker must possess.
This video lecture is aimed to let you know about the Effort Model,
which is at the basis of the skill of listening and speaking simultaneously.
And at letting you learn the skill of listening and speaking simultaneously
in two specific contexts.
Where verbatim accuracy is required,
you will learn to listen and repeat at the same time.
Where sensatim accuracy is required,
you will learn to listen and reformulate at the same time.
This is the agenda of this presentation.
I will first recall the skills of the respeaker.
Then, I will concentrate on the so-called effort model,
at the basis of simultaneous interpreting,
a mental activity, very close to real time subtitling.
I will finally focus on shadowing, a manifold exercise,
you will be requested to do along the course
to develop, reinforce, and finally maintain this skill
in two different contexts: verbatim and sensatim subtitling.
As you have seen,
a respeaker has to do many things at the same time.
In this Element, we will deal with the Psychocognitive skill
of listening and speaking at the same time,
the bulk of the respeaking skills.
In particular, for a respeaker to become a professional,
he or she has to first develop the skill of doing two things at a time.
Listening and understanding the speaker while talking to the computer,
so that real-time subtitles are produced.
You can develop this skill by the time thanks to ad hoc exercises.
But you may be interested in understanding the rationale behind it,
the effort model, and then shadowing as a way to develop it.
So, what is the Effort Model?
In 1985, Daniel Gile proposes the so-called Modèle d'efforts,
or Effort Model,
based on observing the mistakes of trainees in simultaneous interpreting.
Gile realises that trainees do mistakes that are so banal
the reason for these mistakes cannot be a poor command of the language.
He understands that this is due to other reasons,
mainly that our brain is to be trained
if we want it do more than one non-automatic thing at the same time.
indeed, our brain can process only one non-automatic process at a time.
To understand this,
it is important to know that mental activities can be of 3 types.
They can be automatic, like that at basis of breathing.
They can be semi-automatic, like that at the basis of walking,
They can be non-automatic, like that at the basis of active listening
or – guess! – repeating or reformulating what somebody has just said.
And while you can breathe, walk and speak at the same time,
you cannot listen and speak at the same time,
unless you are trained to do so, in courses like this one.
As said in the introduction,
the effort model is thought for Simultaneous interpreting,
but it can be easily adapted to real-time intralingual subtitling.
Adapting Gile’s notions o real-time subtitling,
while respeaking, we mainly do 4 non-automatic actions at the same time.
Listening to the speaker and understanding,
which implies we analyze the waves she produces and turn them into meanings.
Then there is memory,
which is the capacity to temporarily store meaning before we do something with it.
Production is the fact of producing sentences
that need to both adequately render the meanings uttered by the speaker
and be understandable by the Automatic Speech Recognition tool.
Coordination is the fact of monitoring one’s performance,
so that a balance is found between these three efforts.
Now, these efforts require a lot of processing capacity.
As said,
we need to train our brain to find a balance between these four efforts.
As long as our processing capacity is enough
to carry on these mental activities simultaneously,
we then do a good job.
If it is not, because of fatigue or other reasons,
we may experience breakdowns in one of the activities,
with negative consequences on the quality of the subtitles.
In particular, a good performance is possible under these conditions.
First of all, the speaker should speak properly,
not too fast, with a clear voice and not too technically.
Then it is important the respeaker knows the topic or has studied it.
Moreover, the respeaker must work in a well-equipped environment
and with working turns which are not too stressful.
And finally, the interaction with the machine should be at its utmost.
In case one or more of these conditions are not met,
more processing capacity is required
in one of the four efforts to bridge the gap.
This may cause a breakdown in one of the other activities
which, in the end, go to the detriment of the quality of the subtitles.
In particular, if the speaker is challenging,
it will be harder for the respeaker to listen and understand.
If the topic is not well mastered or totally unknown,
it will be harder to remember what the speaker says.
If the respeaker is not in good mental and or physical conditions,
it will be harder to produce coherent subtitles.
Finally, if the interaction with the machine is not good,
coordination breakdowns may occur.
In the next video lectures,
we will see how to cope with each of these situations
that may cause bad performance.
In this video lecture,
we will try and make sure that, in good conditions, you are going to perform well.
Moreover, we will try to see how it is possible to semi-automate
some of the simultaneous, non-automatic actions which are required in respeaking
to reduce the processing capacity needed for each of them.
One way to go for that is shadowing, which we will see in the next section.
In this second section,
we will deal with the notion of shadowing as a tool to develop
and semi-automate the psycho-cognitive skill of listening and speaking at the same time.
Shadowing means you listen to a speech and you repeat it.
There is no need to involve the software for the moment.
However, you should know what it means to repeat.
What should you repeat.
To our understanding, there are four types of shadowing.
Litteratim shadowing, verbatim shadowing, sensatim shadowing and signatim shadowing.
The difference lies in the degree to which you should repeat the source text,
that is the speech you are listening to.
Litteratim shadowing means you repeat every sound you hear of the speaker.
Every speaker, even the most trained one,
makes use of given and recurrent features of orality when speaking.
This means that you should repeat not just words,
but any of them,
even words that you don’t understand the meaning of,
as well as recurrent examples of orality.
These may be fillers
like "well", "you know", "kind of", "you see", etc.
Extra sounds like "uhm", "er", "mh", etc.
Interjections like "Uh!", "Ah!", "Ow!", "Ack!" etc.
Barbarisms, or words
that each of us pronounces his or her own way,
like "heyyy", "naaa", "wassuup", etc.
Verbatim shadowing means
that you should repeat every distinct, meaningful element of the speech you hear
as uttered by the speaker, mistakes included.
This means that you should repeat every single lexical item,
like "house", "government", "Spain", etc.
Grammar items like "the", "when", "for", etc.
Acronyms like "GDP", "NCRA", "OWL", etc.
Foreign words and expressions,
like "ad nauseam", "bon voyage", "gnocchi", etc.
Sensatim shadowing is a bit more complex and less precise as a notion.
It means you should repeat every single meaning uttered by the speaker,
a meaning being any concept expressed as a word,
a clause, or a sentence.
You can either repeat these words as in verbatim shadowing,
or re-express them using other words, normally shorter words.
For example, you can make use of synonyms,
both horizontal synonyms, like "aim" instead of "objective",
and vertical synonyms, like "flower" instead of "anemone".
You can also reformulate the utterance.
For example, you can say "yes" instead of "I tend to agree",
or "we" instead of "you and I".
Finally, you can strategically omit some elements in a sentence
that not necessarily add something to the overall meaning.
For example, instead of saying:
"Carlo talks a lot so as to try and explain this notion",
you can simply say: "Carlo talks a lot to explain this".
Finally, signatim shadowing is the form of shadowing
that is aimed at repeating every single sign,
meaning any meaningful occurrence in a speech event
that adds information to the ideal reader
of the potential transcription of signatim shadowing.
This means not just the single sentences that one hears like "Good morning",
but also any other relevant element such as punctuation marks,
like comma, period, question mark etc., as in "good morning period".
Signatim also means paraverbal events,
like: the tone of voice, intonation, volume etc.,
as in "good morning period [he shouts]".
Signatim finally means non-verbal events,
like applauses, cheering, a bell ringing,
as in "good morning period [applauses]".
In this video lecture we have introduced the main competence of the respeaker,
the psycho-cognitive skill of listening to the ST and simultaneously repeating it.
In particular, we have dealt with two important aspects:
a theoretical one, that is the Effort Model,
and a more practical one, shadowing.
The effort model is at the basis of this cognitive process
that involves doing more things at the same time.
We have seen that balancing the efforts these simultaneous actions require
is always needed to avoid breakdowns.
One way to go for that is trying to semi-automate these actions.
To do so, shadowing is the exercise to start with.
It consists in listening a speech and repeating it,
either litteratim or sound for sound,
verbatim or word for word,
sensatim or meaning for meaning
and signatim or sign for sign.
In the homework session, we will see how to do this in practice.